fix: apply HTTP restrictions to the SageMaker endpoint - #8686
Merged
Conversation
mattwittwer
changed the title
yinggeh
reviewed
Mar 10, 2026
| if (multi_model_name.empty()) { | ||
| LOG_VERBOSE(1) << "SageMaker request: LIST ALL MODELS"; | ||
|
|
||
| if (restricted_apis_.IsRestricted( |
Contributor
There was a problem hiding this comment.
Should create a new macro similar to RETURN_AND_RESPOND_IF_RESTRICTED
…on-inference-server/server into mwittwer/sagemaker_input_size_limit
whoisj
approved these changes
Mar 11, 2026
mattwittwer
added a commit
that referenced
this pull request
Mar 11, 2026
(cherry picked from commit a5b45da)
This was referenced Mar 11, 2026
Contributor
|
This should be feat |
LinZiyuu
added a commit
to LinZiyuu/server
that referenced
this pull request
May 30, 2026
The SageMaker multi-model custom-invoke route (POST /models/{model}/invoke)
is dispatched to SageMakerMMEHandleInfer() without checking the inference
API restriction. When --http-restricted-api=inference is configured, the
standard HTTP infer, the gRPC infer, and the SageMaker /invocations routes
all enforce the required header (the last via the inherited HandleInfer),
but the MME invoke route runs inference without it. This mirrors the MME
model-repository gap closed in triton-inference-server#8686, which did not cover the invoke route.
Add the existing RETURN_AND_RESPOND_IF_RESTRICTED(req,
RestrictedCategory::INFERENCE) guard at the top of the /invoke branch,
matching the base HandleInfer/HandleGenerate handlers and the sibling MME
repository branches. No behavior change when no inference restriction is set.
Signed-off-by: LinZiyuu <linziyu0205@163.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does the PR do?
Implement the core HTTP server’s restricted API and max input size settings in the SageMaker endpoint.
Add regression testing for restricted operations and payload-size enforcement for the Sagemaker endpoint.
Checklist
<commit_type>: <Title>Commit Type:
Check the conventional commit type
box here and add the label to the github PR.
Related PRs:
Where should the reviewer start?
The main changes to pass these parameters are in:
src/main.cc
src/sagemaker_server.h
Then the implementation for the restricted endpoint checks is in:
src/sagemaker_server.cc
Test plan:
45550957
Caveats:
Background
Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)